Image processing apparatus and image processing method

ABSTRACT

It is prevented at a time that a moving image portion is erroneously determined on a screen having a cyclic pattern moving at a constant speed and that edge portions of a still image surrounded by a moving image is subjected to moving image processing. A motion detection portion ( 3, 51 ) for receiving as an input pixel data Pi( 0 ) of an interlaced image and pixel data Pi(+2F) delayed by two fields (2F) thereof, a history value generation portions ( 52, 53 ) for generating a history value (Hk) showing the number of times that determination is continuously made to be “a still image” based on the motion detection result (Dif( 0 )), a pixel data interpolation portion ( 4 ) for mixing pixel data (Pm) by interpolation in a field and pixel data (Ps) by interpolation between a plurality of fields based on pixel data of the interlaced image at a mixture ratio (Rmix) in accordance with the motion detection result (Dif( 0 )) and the history value (Hk) are provided. The larger the history value (Hk) is, the larger amount of pixel data (Ps) by interpolation between fields the pixel data interpolation portion ( 4 ) mixes.

TECHNICAL FIELD

The present invention relates to an image processing apparatus and animage processing method for detecting a motion of an interlaced imageand generating data for complementing line to obtain a noninterlacedimage by a complementing method in accordance with the result.

BACKGROUND ART

Current television signal formats are roughly classified in aninterlaced signal format assuming interlaced scanning and anoninterlaced signal format assuming noninterlaced scanning. Theinterlaced scanning is also called as “skipping” scanning and is amethod of scanning every other line on 525 or 1125 scanning linescomposing one screen of a television image. In this method, one displayscreen (one frame) is generated by interlaced scanning for two times,and one frame is composed of two scanning screens (first and secondfields) having alternating scanning lines. On the other hand,noninterlaced scanning is not “skipping” scanning and is a method ofscanning every scanning line successively.

In an image display device, for example, when displaying other stillimage on a part of a moving image, it is necessary to display the stillimage by converting an interlaced signal to a noninterlaced signal tosuppress flickering on the still image and to attain a high-qualityimage. The conversion is also called as IP (Interlace/Progressive)conversion, and an image display device provided with an IP conversionfunction for multi-screen displaying is known. Hereinafter, conversionof interlace and non-interlace will be called as IP conversion.

Also, depending on kinds of an image display panel, particularly like animage display panel using self-luminous PDP (plasma display panel) andLED (light emitting diode), etc., there are those driven by anoninterlaced progressive signal, and they are provided with an IPconversion function.

There are a variety of IP conversion methods, but a motion adaptive IPconversion method for detecting a motion of an image from difference ofimage data between fields and performing interpolation in a field for amoving image and interpolation between fields for a still image togenerate line data adaptively in accordance with a kind of the image (amoving image or a still image) for obtaining a high-quality image hasbeen widely used. In this method, image data of a new line is generatedby adaptively mixing image data suitable to a moving image obtained byinterpolating from an image in a field as an object of generating linedata (hereinafter, referred to as a moving image interpolation data) andimage data suitable to a still image obtained by interpolating fromimages between two fields including a field as an object of generatingline data (hereinafter, referred to as a still image interpolationdata). When determining the mixing ratio, based on a frame difference ofprevious and following fields of a pixel to be interpolated, a method ofheightening the mixing ratio of moving image interpolation data when theframe difference is large, and heightening the mixing ratio of stillimage interpolation data when the frame difference is small has beenused.

In the motion adaptive IP conversion method, if the determination ismade to be closer to a moving image (that is, determining to heightenthe mixture ratio of moving image interpolation data) when determiningthe mixture ratio, a large failure is not caused to the screen becausethe moving image interpolation is processing in the same field. On theother hand, if a moving image is erroneously determined to be a stillpicture, one screen is created from two fields being different data inaccordance with the motion, contour of the image becomes aliasing,horizontal stripe becomes highly visible or, in bad cases, the imagelooks double, so that failure is caused as a picture. Therefore, in theconventional motion adaptive IP conversion, there was a tendency thatthe determination was made to be close to a moving image.

However, when a still picture is not correctly determined to be a stillimage, a still image having high vertical resolution cannot begenerated, so that the failure as a picture was prevented by sacrificingvertical resolution of a still image in the conventional motion adaptiveIP conversion for determining to be closer to a moving image.

This basic IP conversion method was developed, and there has beenproposed an IP conversion method for using information of more fields,such as six fields, than adjacent two fields so as to reflectinformation of different pixel information of pixels to be interpolatedin terms of time and space (for example, refer to the JapaneseUnexamined Patent Publication No. 2002-185933, which will be referred toas the prior art article 1 below).

In the prior art article 1, as a method of reflecting larger informationof different pixels in terms of time, a difference of intricatelycombined fields in a wide range is calculated, for example, a differenceof current field data and two-field delayed data, a difference ofone-field delayed data and three-field delayed data, a difference oftwo-field delayed data and six-field delayed data, and a difference ofcurrent field delayed data and six-field delayed data. The respectivedifferences are compared with a predetermined threshold value to set aflag, a logical add of the obtained flag data is calculated, and themixture ratio of a moving image and still image is determined based onthe logical add of the flag data. Also, as the case of using informationof spatially different pixels, as described in the above prior artarticle 1, the case of performing interpolation calculation on fourlines above and below a pixel to be interpolated may be mentioned.

In these methods, the mixture ratio is determined by reflectinginformation of different pixels to be interpolated in terms of time andspace, so that, for example, in the case where a cyclic pattern moves atan approximately adaptive speed to the cycle, a moving image part liableto be erroneously detected as a still image because data does not changein micro-scale when seeing for a certain time interval is surelydetected “to be a moving image”. Also, for example, even when letters(tickers), such as alphabets, move on a screen, the case where a movingimage is erroneously detected to be a still image in a part of pixelsdecreases. Accordingly, by using an IP conversion method described inthe prior art article 1, it is possible to prevent deterioration ofimage quality, such that edges of an image look aliasing caused byerroneously detecting a moving image as a still image.

However, in the IP conversion method described in the prior art article1, a large number of fields are referred to for discriminating a movingimage from a still image, and a large capacity field memory isnecessary.

Also, for example, in the case where still tickers created by PC(personal computer), etc. are superimposed to be displayed on a movingimage shot by a camera in a TV program, when a too wide range ofperipheral pixels or a large number of fields are referred, an effect ofsuch wide range of detection results appears as an adverse effect.Namely, there is a possibility that determination to be closer to amoving image is made more than necessary at edge portions of a stillimage (tickers) where the same data should be displayed because thesurround is a moving image. In this case, only the edge portions of thestill tickers look clearer or more blurred than other ticker portions insome cases. Since tickers have higher frequency components in thehorizontal and vertical directions comparing with those in thebackground a moving image, when the edges become blurred, it is highlyvisible and the picture quality declines.

On the other hand, as a method of making edge portions of a still imageless noticeable on a moving image as a background, there is known an IPconversion method including processing of determining a boundary of astill image and a moving image (for example, refer to the JapanesePatent Publication No. 3347234, which will be referred to as a prior artarticle 2).

In this method, based on results of motion detection, in other than thecase where all of three lines: a current line, an output of the previousline and an output of the following line show a motion or all of thethree lines show no motion, it is determined that there is a boundary.Line data is substituted, so that there is no discrepancy between theresult of the boundary determination and the result of motion detection.

The IP conversion method described in the prior art article 2 is anexcellent method in the point that a large capacity field memory is notused and edge portions of a still image can be made less noticeable on amoving image as a background.

However, in the method described in the prior art article 2, informationof a wide range of pixels is not reflected, so that the subject to beovercome by the prior art article 1 explained above, that is, theproblem that a moving image part (a moving cyclic pattern and movingtickers) is erroneously determined to be a still image on a screenhaving a moving cyclic pattern or moving tickers moving at a constantspeed cannot be solved.

DISCLOSURE OF THE INVENTION

An object of the present invention is to prevent a moving image portionfrom being erroneously determined as a still image on a screen having acyclic pattern moving at a constant speed or moving tickers, etc. andedge portions of a still image from being subjected to a moving imageprocessing and becomes highly visible on a moving image as a background.

An image processing apparatus according to the present invention, forconverting an interlaced image data to a noninterlaced image data,comprises: a motion detection portion (3, 51) for comparing pixel dataof an interlaced image (pixel data Di(0) and Di(+2F) comprising fieldscreen Pi(0) and Pi(+2F), hereinafter, be described by referencenumerals of the field screen to which belonging the pixel data inconsideration of correspondence to drawings) to perform a motiondetection; a history value generation portions (52, 53) for generating ahistory value (Hk) showing the number of times that determination iscontinuously made to be “a still image” based on a motion detectionresult (Dif(0)) from the motion detection portion; and a pixel datainterpolation portion (4) for mixing a pixel data (Pm) generated byinterpolation in a field and a pixel data (Ps) generated byinterpolation between a plurality of fields based on pixel data of theinterlaced image at a mixture ratio (Rmix) in accordance with the motiondetection result (Dif(0)) and the history value (Hk), wherein the largerthe history value (Hk) is, the larger amount of pixel data (Ps)generated by interpolation between fields the pixel data interpolationportion (4) mixes.

The pixel data interpolation portion comprises; an in-fieldinterpolation portion (41) for generating the pixel data (Pm) byinterpolation from a pixel data (Pi(+F)) in a filed; an inter-fieldinterpolation portion (42) for generating the pixel data (Ps) byinterpolation from pixel data (Pi(+F) and Pi(+2F)) in a plurality offiled; a pixel data mixing portion (43) for mixing the pixel data (Pm)from the in-field interpolation portion (41) and the pixel data (Ps)from the inter-field interpolation portion (42) at a predeterminedmixture ratio (Rmix); and a mixture ratio setting portion (44) forchanging the mixture ratio (Rmix) determined by the motion detectionresult (Dif(0)) of the motion detection portion (3, 51) and the historyvalue (Hk) in such a way that the larger the history value (Hk) is, thehigher a ratio of the pixel data (Ps) from the inter-field interpolationportion (42) becomes.

An image processing method according to the present invention ofconverting an interlaced image data to a noninterlaced image data,comprises the steps of: motion-detecting by comparing pixel data (Pi(0)and Pi(+2F)) of an interlaced image pixel-by-pixel between frames toperform a motion detection; generating a history value (Hk) showing thenumber of times that determination is continuously made to be “a stillimage” based on a result of the motion detection; and interpolating bymixing pixel data (Pm) generated by interpolation in a field and pixeldata (Ps) generated by interpolation between a plurality of fields basedon pixel data of the interlaced image at a mixture ratio (Rmix) inaccordance with the motion detection result (Dif(0)) and the historyvalue (Hk), wherein the larger the history value (Hk) is, the largeramount of pixel data (Ps) generated by interpolation between fieldsmixes.

The interpolating step of pixel data further comprises; in-fieldinterpolating by generating the pixel data (Pm) of a line having nopixel data in a field by interpolation from pixel data (Pi(+F)) in thefiled; inter-field interpolating by generating the pixel data (Ps) byinterpolation from pixel data (Pi(+F) and Pi(+2F)) in a plurality offiled; mixing of pixel data by mixing the pixel data (Pm) generated bythe in-field interpolating and the pixel data (Ps) generated by theinter-field interpolation portion (42) at a predetermined mixture ratio(Rmix); and setting of a mixture ratio by changing the mixture ratio(Rmix) determined by the motion detection result (Dif(0)) of the motiondetection and the history value (Hk) in such a way that the larger thehistory value (Hk) is, the higher a ratio of the pixel data (Ps)generated by the inter-field interpolating becomes.

In the present invention, as a result of motion detection of pixel dataof an interlaced image by a motion detection portion (3, 51), when thereis no difference or only a small difference in the pixel data, it isdetermined as a still image, while when the difference is large, it isdetermined as a moving image. As to generation of a history value, ahistory value (Hk) as the number of times that determination iscontinuously made to be a still image is generated for each pixel. In apixel data interpolation portion (4) (or in an interpolation step), aninterpolation method of pixels, for which data should be newly created,is determined in accordance with the generated history value (Hk). Indetail, the pixel data interpolation portion (4) comprises an in-fieldinterpolation portion (41) adaptive to a moving image, a inter-fieldinterpolation portion (42) adaptive to a still image, a pixel datamixture portion (43) for mixing outputs of both of the interpolationportions at a predetermined ratio (Rmix), and a mixture ratio settingportion (44) for setting the mixture ratio (Rmix). The mixture ratiosetting portion (44) determines the above mixture ratio (Rmix), so thatthe larger the history value (Hk) is, the closer the interpolationbecomes to a still image, that is, the ratio of pixel data (Ps)generated by the inter-field interpolation becomes high.

In the image processing as above, for example, in the case where arepeating pattern at a predetermined cycle moves at a speed adaptable tothe cycle, at a position of a pixel where the pattern is repeatedlydisplayed, there is a pattern portion always determined to be a stillimage when seeing at a certain time interval. However, the number oftimes that determination continuously made to be a still image for eachpixel, that is a history value (Hk), is counted in the presentinvention, so that the history value (Hk) of the still image becomesintermittent at a cycle corresponding to a width of the pattern.

Namely, when seeing in perspective by using the history value (Hk), therepeating pattern of a predetermined cycle as above is not determined asa complete still image or a complete moving image. Therefore, a mixtureratio (Rmix) adaptive to the repeating pattern cycle or the moving speedis obtained, and new pixel data (Po) is generated at the pixel positionby using the mixture ratio.

Also, when tickers, such as alphabets, move as if it flows, the historyvalue (Hk) becomes intermittent between the letters, so that pixel data(Po) is generated at a position of displaying the tickers by using themixture ratio (Rmix) adaptive to the letter intervals and the movingspeed.

Furthermore, at edge portions of the still image tickers surrounded by amoving image, peripheral pixel information is not reflected and only thehistory value (Hk) at the pixel becomes information for making thedecision, so that the history value (Hk) at the pixel depends on adisplay time of the tickers and determination is made to be sufficientlylarge and closer to a complete still image. Accordingly, new pixel data(Po) is generated by a mixing ratio (Rmix) being close to the stillimage at edge portions of the still image tickers.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an image processing apparatus according toa first embodiment.

FIG. 2 is a view showing a positional relationship of three continuingfield screens.

FIG. 3 is a view showing a method of inquiring a frame difference in apositional relationship of three screens when seeing from the frontsurface A side shown in FIG. 2.

FIG. 4 is a view showing in-field interpolation at a positionalrelationship of three screens in the same way as in FIG. 3.

FIG. 5 is a view showing inter-field interpolation at a positionalrelationship of three screens in the same way as in FIG. 3.

FIG. 6 is a flowchart of history value generation processing.

FIG. 7 is a flowchart of mixture ratio setting processing.

FIG. 8 is a graph showing an example of relationship of two inputparameters with the mixture ratio in a mixture ratio reference table.

FIG. 9 is a view for explaining transition of the mixture ratio when acircular image moves on a still background.

FIG. 10 is a view showing a screen having still image tickers (alphabetletters) surrounded by a moving image.

FIG. 11 is a block diagram of an image processing apparatus according toa second embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

Below, preferred embodiments of an image processing apparatus and animage processing method will be explained with reference to thedrawings. The image processing apparatus is realized as an apparatus oran integrated circuit (IC) having a motion adaptive IP conversionfunction.

FIRST EMBODIMENT

FIG. 1 is a block diagram of an image processing apparatus according toa first embodiment.

An image processing apparatus 1A shown in FIG. 1 roughly comprises afield delay portion 2, a frame difference calculation portion 3, animage data interpolation portion 4 and a history value generationportion 5 of a still image.

The field delay portion 2 comprises a first field delay portion 21 fordelaying an input field screen Pi(0) by one field and outputting thesame and a second field delay portion 22 for delaying a screen Pi(+F)after delaying by one field (1F) input from the first field delayportion by one field. The second field delay portion 22 outputs a fieldscreen Pi(+2F) delayed by two fields (2F), that is, exactly by oneframe.

Here, a display time of one field (or a field screen interval on a timeaxis) is expressed by “F”, a progressed phase is expressed by “+”, and adelayed phase is expressed by “−”. Also, the current point is expressedby “0”. In a state shown in FIG. 1, at the time a field image Pi(0) isinput (current point), a next field screen Pi(+F) input to the firstfield delay portion 21 exactly before one field display time(hereinafter, simply referred to as 1F), wherein the phase is progressedexactly by 1F, is output from the first field delay portion 21. Also,still next field screen Pi(+2F) input to the first field delay portion21 before the current point exactly by two field delay time(hereinafter, simply referred to as 2F), wherein the phase is progressedexactly by 2F, is output from the second field delay portion 22.

Positional relationship of the three field screens Pi(0), Pi(+F) andPi(+2F) is shown in FIG. 2. FIG. 2 shows three dimensionally byintroducing time axis to two dimensional space of the screen, whereinthe time axis is expressed by “→t”. Also, FIG. 3 to FIG. 5 show viewsseeing the positional relationship of the three screen from the frontsurface A side shown in FIG. 2.

It was explained above that an interlace display screen was displayedthrough performing interlaced scanning for two times on a screen (oneframe) completed as a picture and, in FIG. 2, pixels (or pixel data)belonging to a first field screen by the scanning for the first time areindicated by white circles, and pixels (or pixel data) belonging to asecond field screen by the scanning for the second time are indicated byblack circles. In this case, one frame is configured by combining aninput screen Pi(0) at the current point and a screen Pi(+F) wherein thephase is progressed exactly by 1F from that, and another one frame isconfigured by a set of next image having a still progressed phase, thatis, Pi(+2F) and a not shown Pi(+3F). As shown in FIG. 3, the secondfield screen Pi(+F) indicating pixels by black circles and the firstfield screens Pi(0) and Pi(+2F) on its both sides have a positionalrelationship in the vertical direction shifted exactly by one linescanning interval L. Also, since it is interlaced scanning, intervals ofpixel data lines of each screen are set to be two-line scanningintervals (2L).

A frame difference calculation portion 3 shown in FIG. 1 (indicated by“ΔP” in the figure) receives as an input an input field screen Pi(0) atthe current point and a field screen Pi(+2F) after delaying by one framefrom the second field delay portion 22, and obtains, for example, anabsolute value of a difference of luminance data (hereinafter, referredto as a frame difference) for each pixel between frames by calculation.For example, as shown in FIG. 3, a frame difference Dif(0) is obtainedfrom pixel data Dk(0) in the field screen Pi(0) and pixel data Dk(+2F)in the field screen Pi(+2F), and the processing is repeated for eachpixel. The frame difference Dif(0) is successively generated bycalculation for each pixel and input respectively to the pixel datainterpolation portion 4 and the history value generation portion 5.

The pixel data interpolation portion 4 comprises, as shown in FIG. 1, anin-field interpolation portion 41, a inter-field interpolation portion42, a pixel data mixing portion (indicated by “MIX.” in the figure) 43,and a mixture ratio Rmix setting portion 44.

While not particularly illustrated, the in-field interpolation portion41 comprises a line delay portion for delaying input field image datafor each line and an interpolating portion for generating new line databetween lines of an interlaced image by interpolation using delayed linedata, etc. As a result, the in-field interpolation portion 41 cangenerate an interpolation method suitable to a moving image, that is,being capable of newly generating pixel data of a line required for anoninterlaced image only by data in the same field.

The in-field interpolation method is not particularly limited, but, forexample, as shown in an example in FIG. 4, by multiplying 0.5respectively with pixel data Dk(+F) of a focused line and pixel dataDk−2(+F) obtained by delaying the same by one line (two scanning linesin the frame) in a field screen Pi(+F) after delaying by 1F and mixing,new pixel data Dk−1(+F) is generated for an in-between scanning line,which did not have pixel data.

Below, a field screen composed of new pixel data for a moving imagegenerated by a plurality of pixel data in the same field as above isreferred to as “a moving image interpolation screen Pm”.

The inter-field interpolation portion 42 receives as an input two-systempixel data lines (two field screens) wherein the phases are shiftedexactly by one field from each other. While not particularlyillustrated, the inter-field interpolation portion 42 comprises aninterpolation portion for generating new line data between lines of aninterlaced image by interpolation from the input two-system pixel datalines. As a result, the inter-field interpolation portion 42 is capableof newly generating an interpolation method suitable to a still image,that is, pixel data of a line necessary for a noninterlaced image fromdata in different fields adjacent in terms of time.

While the inter-field interpolation method is not particularly limited,for example, as shown in FIG. 5, by multiplying 0.5 respectively withpixel data Dk(+F) of a focused line belonging to the field screen Pi(+F)after delaying by 1F and pixel data Dk(+2F) corresponding to the fieldscreen Pi(+2F) after delaying by 2F and mixing, new pixel data Dk−1(+F)is generated for a scanning line which did not have pixel data on thefield screen Pi(+F) after delaying by 1F.

Below, a field screen composed of new pixel data for a still imagegenerated by a plurality of pixel data belonging to different fields asabove will be called “a still image interpolation screen Ps”.

The pixel data mixture portion 43 receives as an input a moving imageinterpolation screen Pm from the in-field interpolation portion 41,receives as an input a still image interpolation screen Ps from theinter-field interpolation portion 42, successively mixes pixel data ofthe two interpolation screens Pm and Ps by a predetermined mixture ratioRmix determined for each pixel and outputs a new pixel data line (outputscreen) Po. While not particularly illustrated, such a function of thepixel data mixture portion 43 is realized by, for example, twomultipliers for multiplying the respective interpolation data withcoefficients suitable to realizing the mixture ratio Rmix and an adderfor adding outputs of the two multipliers.

The mixture ratio Rmix of the pixel data mixture portion 43 in thepresent embodiment is made changeable. For example, the coefficients ofthe two multipliers are made changeable in the above configuration.

The mixture ratio setting portion 44 as a means for setting and changingthe mixture ratio Rmix controls the mixture ratio Rmix in accordancewith an input history value. The mixture ratio setting portion 44normally receives as an input a frame difference Dif(0) from the framedifference calculation portion 3 and, by using the mixture ratio inaccordance with the frame difference Dif(0) as reference, changes themixture ratio to be the reference in accordance with the input historyvalue H. Controlling of the mixture ratio in accordance with the historyvalue H by the mixture ratio setting portion 44 and provision of thehistory value generation portion 5 are one of significantcharacteristics of the present embodiment.

The history value generation portion 5 compares the input framedifference Dif(0) with the reference REF by the size, and comprises amotion comparison portion 51 (indicated by “COMP.” in the figure) fordetermining a moving image or a still image, a history value memory 52(indicated by “H memory” in the figure) for holding and updating thenumber of times of continuously determining as “a still image” by themotion comparison portion 51 for each pixel, and a history value delayportion 53 for delaying the held history value H by an amount of twofields (2F), that is, an amount of one frame. The history value memory52 has a memory space assigned with an address for each pixel and isconfigured to be capable of incrementing the held data (history value)for each pixel specified by the address. Note that a history value at acurrent point of a focused pixel is indicated by “Hk(0) in FIG. 1 and ahistory value after delaying by 2F is indicated by Hk(+2F).

In detail, the motion comparison portion 51 compares the input framedifference Dif(0) with a predetermined reference REF for determining aboundary of a moving image and still image and, when the input framedifference Dif(0) is a reference REF or larger, determines as “a movingimage, while when the input frame difference Dif(0) is smaller than thereference REF, determines as “a still image”. The motion comparisonportion 51 outputs a signal S51, for example, for outputting ahigh-level pulse every time determination is made to be “a still image”.The history value memory 52 increments and outputs the history valueHk(0) of the stored previous pixel by one frame (having a phaseprogressed by one frame) every time a high-level pulse is outputs due tothe signal S51. The history value Hk(0) is successively delayed exactlyby 2F and a delayed history value Hk(+2F) is successively input to themixture ratio setting portion 44. A content stored in the history valuememory 52 is re-written by a history value Hk(+2F) output to the mixtureratio setting portion 44. Consequently, it is configured that every timedetermination is made to be a still image based on the history valueHk(+2F) output to the mixture ratio setting portion 44, the historyvalue is added one by one. Since the calculation of the history value isoperated based on the frame difference Dif(0), it is preferable that thehistory value is counted at the pixel Dk(0) or Dk(+2F) shown in FIG. 3at this point. On the other hand, as shown in FIG. 5, the pixel Dk(+2F)is used by inter-field interpolation and the pixel Dk(0) is not used. Inthat point, it is preferable to count the history value at the pixelDk(+2F), and the history value Hk(0) is delayed by 2F to be used forsetting a mixture ratio in the present embodiment.

Note that the motion comparison portion 51 and the frame differencecalculation portion 3 compose an embodiment of “a motion detectionportion” of the present invention.

FIG. 6 is a flowchart of the history value generation processing.

At a point of starting the processing shown in FIG. 6, it is assumedthat the history value stored in the history value memory 52 is Hk(−2F).In a step ST1, a history value obtained by delaying by one frame, thatis, two fields (2F) in the previous processing, that is Hk(0) is inputto the history value memory 52.

When a frame difference Dif(0) is input in a step ST2, the framedifference Dif(0) is compared with the reference REF in the next stepST3. When the frame difference Dif(0) is the reference REF or larger,determination is made that the pixel belongs to a moving image and thehistory value Hk(0) of a pixel stored at an address corresponding to thehistory value memory 52 is reset in a step ST4A. As a result, an imageportion to be processed is determined to become out of the still stateand entered to a moving image state. On the other hand, when the framedifference Dif(0) is smaller than the reference REF, it is determinedthat the pixel belongs to a still image, the history value Hk(0) of astill image stored in an address corresponding to the history valuememory 52 is incremented and it is determined that a still image statecontinues at an image portion to be processed in a step ST4B.

After that, in a step ST5, the history value Hk(0) is delayed exactly by2F by the history value delay portion 53, and a delayed history valueHk(+2F) is sent to the mixture ratio setting portion 44, and a contentin a memory region at an address corresponding to the pixel in thehistory value memory 52 is re-written by the value of the delayedhistory value Hk(+2F).

This processing is performed repeatedly every time a frame differenceDif is input for each pixel.

FIG. 7 is a flowchart of mixture ratio setting processing.

When the mixture ratio setting portion 44 receives as an input a framedifference Dif(0) in a step ST10 and receives as an input a historyvalue Hk(+2F) corresponding to a frame difference Dif(0) in a step ST11,it obtains a mixture ratio Rmix of a moving image interpolation screenPm and a still image interpolation screen Ps in the next step ST12. Thecalculation may be operated point by point, but a table for specifying amixture ratio by two input parameters: a frame difference and a historyvale is incorporated here, and the mixture ratio Rmix is obtained byreferring to the table.

FIG. 8 shows a graph of an example of a relationship of the two inputparameters and a mixture ratio in the table.

Conventionally, the mixture ratio was determined only by a framedifference regardless of the number of times that determination is madeto be a still image but, in the present embodiment as an example shownin FIG. 8, the mixture ratio is determined to be constant regardless ofa frame difference when the history value is smaller than a certainvalue. Note that the graph is just an example and there are a variety ofmethods for determining the mixture ratio. When seeing in perspective,the larger the history value, the closer to a still image the mixtureratio is determined when the frame differences are the same; and thesmaller the frame difference is, the closer to a still image the historyvalue is determined when the history values are the same. Here, when theframe differences are the same, it is not always the case that thehistory value always has to be close to a still image when the historyvalue is large. As in the example in FIG. 8, a portion where a mixtureratio does not change even when the history value changes may bepartially included. “When seeing in perspective” means the mixture ratiochanges along with the history value in a long span.

Note that “the larger the history value is, the closer to aninterpolation method of a still image an interpolation method ischanged” in the present invention includes the case of seeing inperspective as above. Namely, in the present invention, other than thecase where the larger the history value is, the closer to a still imagethe interpolation method gradually becomes, there is a part where aninterpolation method does not change even when the history value changesin the middle. But when seeing in perspective, the case where theinterpolation method changes closer to a still image as the historyvalue becomes larger is also included.

Note that FIG. 8 shows examples of values of a mixture ratio of a stillimage. In this case, Rmix=0 in the case of a complete moving image,Rmix=1.0 in the case of a complete still image, and the Rmix value maybe set close to 1 as it gets closer to the still image between them.

In the present embodiment, when determining a mixture ratio of a movingimage interpolation screen Pm and a still image interpolation screen Psbased on a frame difference Dif, by referring to a history value as thenumber of times that determination is continuously made to be a stillimage, it is possible to refer to a motion state of a large number offields in the past in the same way as in the case of using a largenumber of field delay memories. Therefore, determination of a movingimage or a still image becomes assured. Also, since the mixture ratio ofinterpolation screens can be more gently changed, so that it is possibleto prevent that an image immediately after becoming still is abruptlysubjected to still image processing and looks as if the resolutionabruptly improves.

Furthermore, a threshold (reference REF) of still image history used atthe time of calculating a history value of a still image shown in FIG. 8can be set sufficiently large comparing with noise components of a framedifference, so that the history value is hard to be affected by a noiseof a frame difference.

The parameter called a history value introduced in the presentembodiment indicates that the larger the value, the higher thepossibility of being a still image. Accordingly, when the history valueof a still image is large, determination can be made to be closer to astill image comparing with determination only by a field difference. Asa result, an effect of converting motion adaptive interlace andnon-interlace can be improved in accordance with a variety of cases.

For example, when a pattern repeating by a predetermined cycle moves ata speed adaptive to the cycle, there is a pattern portion alwaysdetermined to be a still image when seeing by a certain time interval ata pixel position where the pattern is repeatedly displayed. However,since the number of times (history value) that determination to be astill image is counted for each pixel in the present embodiment, thehistory value of the still image becomes intermittent at a cyclecorresponding to a width of the pattern. Namely, since the history valueis reset when it reaches a certain value, when seeing in perspective byusing the history value, the predetermined cyclic repeating pattern isnot determined to be a complete still image nor a complete moving image.Therefore, a mixture ratio becomes adaptive to the cycle of therepeating pattern and the moving speed, and new pixel data is generatedat the pixel position by the mixture ratio. Also, as shown in FIG. 9,even when determination is made close to a still image at first, thereis an advantage that determination is made close to a moving image whena time that a certain pattern moves is long.

Also, when tickers, such as alphabets, move as if they flow, a historyvalue between the letters becomes intermittent, so that pixel data isgenerated at a position of displaying the tickers at a mixture ratioadaptive to the letter intervals and moving speed in the same way.

Furthermore, as shown in FIG. 10, at edge portions of still imagetickers (alphabet letters) surrounded by a moving image, periphery pixelinformation is not reflected and only a history value at the pixelbecomes information for making decision, so that the history value atthe pixel depends on a displaying time of the tickers and becomessufficiently large, and determination close to a complete still imagecan be made. Accordingly, new pixel data is generated at a mixture ratioclose to a still image at the edge portions of the still image tickers.

SECOND EMBODIMENT

The present embodiment relates to a change of a history value generationportion.

FIG. 11 shows a block diagram of an image processing apparatus accordingto a second embodiment.

A different point of the image processing apparatus 1B from the imageprocessing apparatus 1A shown in FIG. 1 (first embodiment) is that thehistory value delay portion of the history value generation portion 5 isdivided to a first history value field delay portion 54 for delaying byone field and a second history value field delay portion 55 and, fromthe intermediate connection point, a history value Hk(+F) after delayedby 1F is output. The history value Hk(+F) after delaying by 1F is inputto a mixture ratio setting portion 44 together with a history valueHk(+2F) after delaying by 2F output from the second history value fielddelay portion 55.

As a result, the mixture ratio setting portion 44 in the presentembodiment also refers to a pixel used for interpolation in a 1F delayscreen Pi(+F), for example, in the example shown in FIG. 4, a historyvalue Hk(+2F) of a pixel Dk(+F) and/or Dk−2(+F). Therefore, moresophisticated and delicate determination becomes possible.

In the first embodiment, when a history value was Hk(+2F) with the sameframe difference Dif, the mixture ratio Rmix was determined to be one,while in the present embodiment, the mixture ratio can be controlled tobe delicately changed furthermore by a history value Hk(+F). Also, forexample, when history values Hk(+F) of pixels Dk(+F) and Dk−2(+F) shownin FIG. 4 are both “0”, in other words, it is determined there is hightendency that the both are a moving image. Since a pixel to beinterpolated between the pixels is sandwiched by a moving image pixelsfrom above and below even if it is determined to be a still image basedon a history value shown in the first embodiment, it is possible to adddetermination of outputting a mixture ratio close to a moving image. Asexplained above, in the second embodiment, a variety of determinationcan be made by combining a larger number of history values.Consequently, highly reliable motion adaptive controlling becomespossible.

Note that in the first and second embodiments, a frame difference isalso obtained between pixels adjacent in the direction (scanningdirection) of an arrow shown by “B” in FIG. 2, so that accuracy can beheightened in determination of a moving image and a still image. Also,by counting a history value in pixels adjacent in the scanningdirection, accuracy of the history value can be also heightened.

1. An image processing apparatus for converting an interlaced image datato a noninterlaced image data, comprising: a motion detection portion(3, 51) for comparing pixel data of an interlaced image (pixel dataDi(0) and Di(+2F) comprising field screen Pi(O) and Pi(+2F),hereinafter, be described by reference numerals of the field screen towhich belonging the pixel data in consideration of correspondence todrawings) to perform a motion detection; a history value generationportions (52, 53) for generating a history value (Hk) showing the numberof times that determination is continuously made to be “a still image”based on a motion detection result (Dif(0)) from the motion detectionportion; and a pixel data interpolation portion (4) for mixing a pixeldata (Pm) generated by interpolation in a field and a pixel data (Ps)generated by interpolation between a plurality of fields based on pixeldata of the interlaced image at a mixture ratio (Rmix) in accordancewith the motion detection result (Dif(0)) and the history value (Hk),wherein the larger the history value (Hk) is, the larger amount of pixeldata (Ps) generated by interpolation between fields the pixel datainterpolation portion (4) mixes.
 2. An image processing apparatus as setforth in claim 1, wherein said pixel data interpolation portion (4)comprises; an in-field interpolation portion (41) for generating thepixel data (Pm) by interpolation from a pixel data (Pi(+F)) in a filed;an inter-field interpolation portion (42) for generating the pixel data(Ps) by interpolation from pixel data (Pi(+F) and Pi(+2F)) in aplurality of filed; a pixel data mixing portion (43) for mixing thepixel data (Pm) from the in-field interpolation portion (41) and thepixel data (Ps) from the inter-field interpolation portion (42) at apredetermined mixture ratio (Rmix); and a mixture ratio setting portion(44) for changing the mixture ratio (Rmix) determined by the motiondetection result (Dif(0)) of the motion detection portion (3, 51) andthe history value (Hk) in such a way that the larger the history value(Hk) is, the higher a ratio of the pixel data (Ps) from the inter-fieldinterpolation portion (42) becomes.
 3. An image processing apparatus asset forth in claim 1, wherein said history value generation portions(52, 53) generates a history value (Hk(+2F)) for interpolation of anadjacent pixel in a field delayed by one field from a field where apixel data to be generated by the interpolation and updates with respectto each interpolation.
 4. An image processing apparatus as set forth inclaim 1, wherein said history value generation portions (52, 53)generates a history value (Hk(+F)) for an interpolation of an adjacentpixel in a field differing from a field where a pixel data to begenerated by the interpolation, generates a history value (Hk(+2F)) foran interpolation of an adjacent pixel in the same field where a pixeldata to be generated by the interpolation, and updates respectively withrespect to each interpolation.
 5. An image processing method ofconverting an interlaced image data to a noninterlaced image data,comprising the steps of: motion-detecting by comparing pixel data (Pi(0)and Pi(+2F)) of an interlaced image pixel-by-pixel between frames toperform a motion detection; generating a history value (Hk) showing thenumber of times that determination is continuously made to be “a stillimage” based on a result of the motion detection; and interpolating bymixing pixel data (Pm) generated by interpolation in a field and pixeldata (Ps) generated by interpolation between a plurality of fields basedon pixel data of the interlaced image at a mixture ratio (Rmix) inaccordance with the motion detection result (Dif(0)) and the historyvalue (Hk), wherein the larger the history value (Hk) is, the largeramount of pixel data (Ps) generated by interpolation between fieldsmixes.
 6. An image processing method as set forth in claim 5, whereinsaid interpolating step of pixel data further comprises; in-fieldinterpolating by generating the pixel data (Pm) of a line having nopixel data in a field by interpolation from pixel data (Pi(+F)) in thefiled; inter-field interpolating by generating the pixel data (Ps) byinterpolation from pixel data (Pi(+F) and Pi(+2F)) in a plurality offiled; mixing of pixel data by mixing the pixel data (Pm) generated bythe in-field interpolating and the pixel data (Ps) generated by theinter-field interpolation portion (42) at a predetermined mixture ratio(Rmix); and setting of a mixture ratio by changing the mixture ratio(Rmix) determined by the motion detection result (Dif(0)) of the motiondetection and the history value (Hk) in such a way that the larger thehistory value (Hk) is, the higher a ratio of the pixel data (Ps)generated by the inter-field interpolating becomes.